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cases. A double cross-validation method was used to test the 
stability of the canonical correlations. The magnitudes and shrinkage 
of the largest and the mean significant canonical correlations were 
compared by means of MANOVA across the different decision rules^ 
built in correlation structures, number of starting variables, and 
number of cases. (Author) 



A ste pwise canonical procedure, includ ing tvo 
in d ices for variable deletion and a rule for stopping 
procedure, was derived as a method of select ing core 
from predictors and criteria. The procedure was applied 
data varying in the degree of built in structures in 
correlation matrices, nuiaber of variables, and number of 
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INTRODUCTION 

Educational research in natural settings is inherently multivariate 
in structure and as such involves numerous related variables and correlated 
criteria. Selection and placement problems of a school, a firm, or an 
industry are cases in point, as is the evaluation of educational programri 
and policy. However, in most instances it is not practicable to take 
all possible predictors and criteria into consideration. The researcher 
usually has to select restricted numbers of variables from each of 
pi^edictor and vriterion variable sets, even when he/she uses multivariate 
research designs. 

The primary concern of the present study was to develop an analytical 
stepwise canonical procedure that could be used to systematize the selection 
of core variables from two sets of variables, namely/ predictors and criteria 
The procedure was designed to select those core variables which could explain 
in the most parsimonious way, the relationships between the two sets of 
variables. It was also hoped that the core variables selected by the pro- 
cedure would provide users with more stable statistics, as tested by the 
cross-validation method, than the use of whole sets of variables. 



This study is a part of the dissertation which was submitted 
in partial fulfillment of the requirements for the degree of Ph.D. 
in Education in the Graduate College of the University of Illinois 
at Urbana-Champaign, 1972. The author would like to extend his 
gratitude to Dr. M. M. Tatsuoka, academic advisor, and his 
dissertation committee members for all the support they rendered 
for this study. 



At the same time, the present study aimed to compare the magnitudes of 
canonical correlations and shrinkage in relation to the proportion of vari- 
ables retained, across different built-in population correlation structure 
groups, the number of starting variables, and the number of cases (sample 
size) ♦ 



ANALYTICAL PROCEDURE 

The problem of deriving an analytical procedure involved the develop- 
ment of an optimum selection procedure, a selection index, and a stopping 
rule. A backward elimination procedure was derived for this study. Four 
procedures were examined as potential sources for deriving a selection 
index: the correlation of each variable with its canonical-variate (simi- 
lar to Huberty's study, 1971); the principal components of both predictor, 
and criterion correlation matrices; the multiple correlation of each vari- 
able with the variables in the other variable set (Hall, 1971^; and eigen- 
vectors obtained from canonical analysis. The two selection indices 
chosen lor use in this study were derived via the eigenvector approach. 

Selection Index T 

As Roy (1957) observed, the largest eigenvalue and its associated 
eigenvector play Important roles in mo..c multivariate analyses. The 
weight vectors (u and v ^^) associated with the largest eigenvalue (the 
square of the largest canonical correlation, o ) were used as criteria 
for selecting a variable to be deleted. In other words, the variable 
that had the smallest absolute value was deleted at each stage of the 
variable elimination procedure. 
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Selection Index II 

The second Index was derived from a weighted sum of the absolute values 

of vectors that were associated with significant cinonlcal correlations. 

r . ^ 

The values I !p.^ u, . I and l \o.^ v .J were computed for each of 

i=I 1 kl • i=l . ' 

the predictors and criteria, respectively, where r represented the number of 
significant canonical correlations as tested by Bartlett's method. 
The variable that had the smallest value was deleted at each stage of the 
stepwise analysis. 

A combined criterion of rational and arbitrary criteria was established 
as a rule for deciding when to stop the process of variable deletion. The 
stepwise procedure for variable elimination was terminated when the deletion 
of a variable led to the satisfying of either one of the stopping rules 
described below. At that point, the last variable deleted was restored 
into its variable set. 
Stopping Rule 1 

For practical purposes, the first stopping rule was derived from 
Bartlett*s (1941) x' approximation. In the significance testing of canonical 
correlctions of p predictors and q criteria, 

- ^ V - ^ (p + q + 1) 1 In A 0) 

is distributed approximately as with pq degrees of freedom. Then ono 
variable is removed either from predictors or from criteria, 

- ( V - ^ (p» + + 1) } In ' ' (2) 

is now distributed as x'^ with p'q* degrees of freedom, where p^q* is eitlier 
(p - l)q or p(q -1), so that 



(x'-pq - x'p'q')/(pq - p'q') 



(3) 



x-^ P'q'/p'q' 



is at least approximately distributed as an F-variable with (pq - p'q') 
and p'q' degrees of freedom for the numerator and denominator, respectively. 
If the F-value exceeded a predetermined value, such as p<.05, the stepwise 
procedure of variable deletion was terminated. 

In the same way the significance of the decrease of significant 
canonical correlations, resulting from the deletion of one variable, was 
tested by computing the difference of two x'' differences and their degrees 
of freedom; 

(Xpq - X(p-r)(q-r)) " {Xp-q- " X^p ' -r) (q '-r ) ^ 



tX%-q- - X^p'-r)(q'-r)) / + q' " O 

with 1 and (p' + q' - r) degrees of freedom for the numerator and denomina- 
tor, respectively, and where r repr.-sentcd the number of significant canoni- 
cal correlations. 
Stopping Rule 2 

Hotelling (1936) has indicated that Wilks' lambda, defined by 

q 

A n (1 - Pi^) , 

i°l 

is an index of alienation. Then (1-A) represents the variance accounted 

for by canonical correlations. The ratio, 

^ ~ ^p'q' (6) 

Hhowfl Ihc proportion of variance acNounted for by the r.Hkiceil Hot o! 
variables aH compared to that account od for by tlie whole set of v/irlabh'H. 



HoiKO, the stt'pwise^ varlablo uoleLion proccnlure was lorminaLtHl when 

PJ- < (1 - a) . say, .90 . (7) 



1 - Apq 



Cross- V a 1 i da t ion 

The stability of sample canonical correlations was tested by a 

cro.ss-valicat ion method. The cross-validated canonical correlations were 

computed by elementwise division of. the diagonal elements of — xy^ 

— the' numeialor — by the corresponding square root of the elementwise 

t 

product of a diagonal element of T ^"xk^ -1 — ^ -1 — ^ 

drnoni n<u or ; 

^l' 5xy2 XI. 

j.^^ 3 



Aui/ 1^x2 ^.ii> (y-i'i ^^yy2 yJi> 



(8) 



where the subscript "T' represents the derivation sample, and the subscn'ot 

the validation sample. A double cross-validation rae^'-^od was used; 

th<it is, the weights derived from one sample were cross-validated on the 
other sample, and vice versa. 



DATA AND MKTHODS OF COMPARISON 

The analytical procedure was applied to a Si^ries of simulated data 
varying in degree of built-in structure's in the form of population 
correlation matrices, number of starting variables, and number of cases 
(sample size). Two varying factors were built in the structures of popula 
tion correlation matrices? the .uunbcr of clusters in the predictor 

and criterion sets (O-O, 0-1, 1-1, 1-2, arid 2-2 cluster groups); and, high 
or low predictor-criterion intercorrel at ions. For convenience, the same 
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number of variables were used as predictors and criteria at the beginning. 

Cases with 6, 9, and 12 variables for each of the predictor and criterion 

sets were investigated in this studv. Thus, thirty-six different population 

correlation matrices were generated' for this study. 

From each population correlation matrix, two small and two large sample 

correlation matrices were generated by using Kaiser-Dickman' s method (1962) 

in order to contrast small and large sample cases, and for cross-validation 

purposes (144 sample correlation matrices in total). 

The magnitudes of resulting canonical correlations and skrinkage, and 
the proportion of variables retained were compared by using 3X3X6X2X2 
fixed factorial design multivariate analysis of variance across: variable 
deletion methods (3 levels: Method I = use of whole set of variables; 
Method II » use of Selection Index I; and Method III = use of Selection 
Index II), the initial numbers of variables (3 levels: 12, 18, and 24 vari-- 
able groups), the six built-in population correlation structure groups, the 
high and low predictor-criterion intercorrelation groups, and small and large 
sample groups. The MANOVA was performed for the largest canonical correlation 
and its shiinkage and for mean significant canonical correlations .and mean 
shrinkage, separately. Canonical correlation coefficients were transformed 
Into Fisher's 7/ s and proportions of variables retained were converted into 
radians by an arcsine transformation before the MANOVA was performed (l-dwards, 

9 

1968) .. 

Data generation, canonical analyses, cross-validation, data transforma- 
tion and data analvBla were all processed by TRM ^r>0/5n-7S at the University 
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of Illinois by using the SOUPAC and UMaVAC programs. 

FINDINGS AND DISCUSSIONS 

About 36 percent of the variables were eliminated by using Selection 
Index I, and approximately 38 percent by Selection Index II. The deletion 
of variables brought about considerable decrease in the magnitudes of the 
largest and mean significant canonical correlations. However, the magni- 
tudes of shrinkage were significantly smaller for the reduced sets of vari- 
ables than for the whole sets of variables. (The difference between the 
shrinkage of the largest canonical correlation of the reduced sets of 
variables by Selection Index I and that of the whole sets of variables, 
however, was not statistically significant.) 

Therefore, it is concluded that the elimination of lesser contri- 
buting variables by either Selection Index I or Selection Index II results 
in more stable canonical correlation. It still remains to be determined if 
the sample canonical correlations obtained by the present stepwise procedure 
are more stable than those obtained by any random selection method. 

When the canonical correlations resulting from the use of Selection 
Index I were compared with those resulting from the use of Selection Index 
II, the former were slightly larger than the latter. A few more variables 
were deleted in the latter case. The magnitudes of shrinkage of the largest 
and mean significant canonical correlations of the Selection Index T group 



Department of Computer Science, University of Illinois, SOUPAC program 
descriptions; Statistically Oriented Users Programming and Consulting, 
1972. The UMAVAC was originally developed by J. D. Finn and adapted 
for use at the University of Illinois by J. L. Wardrop and T, J. Blish. 
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were also larger than those of the Selection Index II group, although the 
differences were not statistically significant. Most other conip.ir isons 
between Selection Index I and Seleclion Index II showed similar results. 
Hence, it seems that the strategy of choosing a selection index sliould 
depend upon the user's intention. Selection Index I caused premature 
termination when the variable to be deleted had large weights on signi- 
ficant canonical-var iates other than the first one. This was especially 
true when the deleted variable had large weights on the second and the 
third variates. Selection Index I is recommended when the user is inter- 
ested only in uhe largest canonical correlation and when he can be satis- 
fied with a rough sieving of valriables. Selection Index II 'quires more 
complicated computations, but ^ives more stable sample stotistics with 
fewer variables, ^ 

One of the most important sources of variation in both univariate and 
multivariate analyses of variance is the sample size factor. In the pre- 
sent study, it was the dominating source of variation in the analyses of 
mean significant canonical correlations and shrinkage. Apparently more 
variables were deleted from the small sample group (43%) than from the 
large sample group (32%). However, the magnitudes of the resulting largest 
and mean significant, canonical correlations of the small sample group were 
still larger than those of the large sample group. The difference between 
the means of mean significant canonical correlations of the small and large 
sample groups was prominent because the number of significant canonical 
correlations increased as the number of cases increased. The amounts of de- 
crease in canonical correlations caused by the deletion of variables of the 
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small sample group were evidently larger than those of the large sample 
group, mostly because more variables were eliminated from the former group. 

The shrinkage in the small sample group was also larger than that in 
the large sample group. When the decremental tendencies of shrinkage were 
compared for each sample size group, the decreases in shrinkage due to 
deletion of variables were conspicuous for the small sample group, but 
there was no noticeable change for the large sample group. This implies 
that when sample size is small, the proposed variable elimination procedures 
will be more effective in achieving more stable pairs of canonical-variates . 

The predictor-criterion intercorrelation factor was another important 
source of variation. Noticeably more variables were removed from the high 
intercorrelation group (48%) than from, the low intercorrelation group (28%). 
The magnitudes of the largest and mean significant canonical correlations of 
the high intercorrelation group were still larger than those of the low 
intercorrelation group. The amounts of decrease in the largest and mean 
significant canonical correlations due to the variable elimination were signi-- 
ficantly larger for the high intercorrelation group than for the low inter- 
correlation group, because more variables were deleted from the former group. 

On the other hand, the magnitudes of shrinkage in the high intercorre- 
lation group were smaller than those in the low intercorrelation group. The 
same tendency remained after deletion of variables. Consequently, it can be 
said that the present variable-^deletion procedures are applicable to both high 
and low intercorrelation cases, and they are more efficient in the sense that 
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they give more stable sample statistics wltli fewer variables whon the 
predictor-criterion intercorrelations are large. 

The number of starting variables was the next most important factor in 
the present study. Average percentages of the eliminated variables were 
31, 35, and 47 percent for the groups starting with 12, 18, and ?A variables, 
reapttct ively . When the proportions of the eliminated variables are converted 
into the actual number of variables, four varlablt^s on the average were 
deleted from the group starting with 12 variables, six from the group start- 
ing with 18 variables, and eleven fro»n the group starting with 2A variables. 
The largest and mean significant canonical correlations of the whole set of 
variables of the group starting with 18 variables (.982 and .840) were signi- 
ficantly larger than those of the groups starting with 12 (.925 and .834) and 
24 variables (.967 and .817). The moan of the largest canonical correlation 
was originally the smallest for the group starting, with 12 variables, and 
Willie the mean of significant canonloal correlation means was the smallest 
for the group starting with 24 varl.^hles. The vaiiable deletion did not 
affect tiie relative standings of the largest and mean significant canonical 
correlations of these groups. However, the shrinkage for the original whole 
set ot variables of the group starting with 12 variables was significantly 
larger than those for the gri^ups starting with 18 and 24 variables. Generally, 
deloLion of variables caused decrease in amounts if shrinkage. The dov-llning 
rate was the sharpest for the group starting with 24 variables and the mildest 
for the group starting with 12 variables. Accordinglv, it is concluded that 
the proposed variable-elimination methods an* mori' usofui when the numhor of 
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